The City of South Bend’s Department of Innovation & Technology serves the community by leveraging technology, data, and design to empower the city’s workforce and improve the quality of life for those in the region.
This project focuses on helping the department evaluate data collected from the American Community Survey, the South Bend Community Survey, and 311 call logs. The city has already evaluated these data sets individually and currently presents insights from this analysis within interactive visualizations on their website. Our task is to continue this exploration of the data sources, but to expand on it by aggregating these data sets to generate new insights.
The City would like to understand if they are fulfilling the needs of their residents in their day-to-day lives. Insights will be generated to provide resolution on the district level to evaluate areas of the city that can be highlighted for future investment or evaluated for efficacy of instated programs.
The city hopes to understand the story that these three data sets can tell when evaluated together and will use insights from this evaluation to assess and improve resident services, city outreach, and equity among residents and neighborhoods. Below are the particular insights that are investigated in this report.
Before we begin our analysis, we first have to load in all of our relevant data sources and aggregate them together to a workable format. This became quite a lot of code, so we decided to intentionally hide this code from our final report but it is included in the R Markdown version so that you can easily reproduce all of the data cleaning done for our work.
The 311 Call Center (311.southbendin.gov) is responsible for facilitating citizens’ needs through various city services. Any South Bend resident calls 311 to address issues and needs including trash pickup, reporting potholes, water quality issues, and much more. All of the data on 311 calls is tracked in the city’s citizen relationship management (CRM) system in various tables that include information on the phone call, knowledge interactions (the topic of the call), or service requests that may result from the call. Below are some basic definitions and summary statistics about the 311 call data.
The following plot displays the volume of 311 calls and the percentage that resulted in a service request from 2018 to 2022.
The 311 call volume has decreased over the years. However, it is important to note that the percentage of 311 calls resulting in service requests has increased.
Below displays the total number of service requests by Census Tract. Use the toggle to the right to filter to a particular Census Tract and view their total number of service requests fulfilled over time.
The South Bend Community Survey (https://southbendin.gov/communitysurvey) is a biannual survey administered by community leaders to assess resident perceptions, priorities, and satisfactions. The surveys are anonymously collected via mail, phone, and online. The survey aggregates 11 different questions with a total of 56 total sub-questions to assess resident satisfaction, investment preferences, and city resource interaction.
The following city services are assessed in the community survey.
The following plot displays the number of responses to the South Bend Community Survey for the last three surveys (2018, 2020, 20022).
For the past 3 surveys, the number of responses has been relatively consistent, ranging from 639 to 682 per year.
In the survey, the satisfaction of different city services is measured on a likert scale, where 1 represents being extremely unsatisfied and 5 represents being extremely unsatisfied with a city service. Below displays the average satisfaction for a subset of city services measured in the last three community surveys.
It is important to note that for the last six years, overall satisfaction has remained relatively steady. The satisfaction is trash services has increased over time, and the satisfaction with city planning and local enforcement have decreased over time.
Below displays the satisfaction for the different City Services measured in the South Bend Community Survey for the last 3 surveys by Census Tract. The toggles to the right filter the plot to examine the satisfaction by a particular Census Tract and City Service.
The United States’ Census Bureau (data.census.gov) collects annual survey data to analyze social, economic, and geographic conditions of the United States, states, and counties. One of these surveys, the American Community Survey (ACS) helps local officials, community leaders, and businesses understand the changes taking place in their communities. It is the premier source for detailed population and housing information about our nation. The following tables from the 2020 ACS were used for this analysis.The tables were tabulated by Census Tract in St. Joseph County, Indiana.
Tables:
Upon investigating the address provided by community survey respondents, a map was generated to share areas of the city where respondents are located. In total, survey responses were captured in 48 out of the 81 census tracts comprising St. Joseph County.
| Census Tract | Count Respondents |
|---|---|
| 32 | 33 |
| 118.03 | 31 |
| 11 | 29 |
| 35 | 26 |
| 2 | 25 |
| 31 | 24 |
| 1 | 23 |
| 34 | 23 |
| 10 | 22 |
| 26 | 22 |
| Census Tract | Count Respondents |
|---|---|
| 111 | 1 |
| 112.03 | 1 |
| 115.01 | 1 |
| 117.04 | 1 |
| 19 | 1 |
| 113.09 | 3 |
| 117.03 | 4 |
| 17 | 4 |
| 27 | 5 |
| 110.01 | 7 |
| 5 | 7 |
| 7 | 7 |
The map below displays the locations of 82,279 total service requests placed in the City. Service requests were made in 71 census tracts, with tract 31 leading with 4,325 unique requests.
Here are the top 10 Census Tracts based on the number of Service Requests:| Census Tract | Service Requests |
|---|---|
| 31 | 4325 |
| 21 | 4018 |
| 2 | 3747 |
| 32 | 3690 |
| 34 | 3525 |
| 14 | 3468 |
| 4 | 3004 |
| 15 | 2892 |
| 33 | 2861 |
| 11 | 2671 |
| Census Tract | Service Requests |
|---|---|
| 102.02 | 1 |
| 108 | 1 |
| 114.03 | 1 |
| 114.06 | 1 |
| 115.04 | 1 |
| 115.05 | 1 |
| 115.06 | 1 |
| 116.02 | 1 |
| 124 | 1 |
| 418 | 1 |
Next, to acknowledge that the population of each census tract is varied, we seek to understand the leading tracts using 311 as a function of each tract’s population. To achieve this, we read in a file from the Decennial Census of 2020 that lists the populations for each census tract. Below, we show the bottom 20 census tracts by calls per capita.
Additionally, we have plotted calls per capita on the county map; we immediately notice that Census Tract 21 has the highest number of calls per capita at 3.36 calls per capita. While this is quite an interesting data point, we omit this tract and regenerate the city map to evaluate the other tracts on their own scale due to the magnitude of Tract 21’s call volume.
From the omitted county map, we see that the neighboring districts also show a relatively high call volume per capita. Census Tracts 4 and 5 are next highest, at 1.33 and 1.35 calls per capita, respectively.
This is the top 10 Census Tracts based on Calls per Capita:| Census Tract | Calls Per Capita | Number of Calls | Population |
|---|---|---|---|
| 21 | 3.365159 | 4018 | 1194 |
| 4 | 1.333925 | 3004 | 2252 |
| 5 | 1.315060 | 2183 | 1660 |
| 19 | 1.275336 | 1422 | 1115 |
| 6 | 1.239382 | 2568 | 2072 |
| 30 | 1.204748 | 2030 | 1685 |
| 31 | 1.180082 | 4325 | 3665 |
| 2 | 1.093695 | 3747 | 3426 |
| 9 | 1.054920 | 1383 | 1311 |
| 14 | 1.035842 | 3468 | 3348 |
| Census Tract | Calls Per Capita | Number of Calls | Population |
|---|---|---|---|
| 116.02 | 0.0001362 | 1 | 7340 |
| 114.03 | 0.0001688 | 1 | 5924 |
| 114.06 | 0.0001933 | 1 | 5172 |
| 115.06 | 0.0002019 | 1 | 4953 |
| 108 | 0.0002146 | 1 | 4660 |
| 115.05 | 0.0003326 | 1 | 3007 |
| 114.04 | 0.0004357 | 3 | 6886 |
| 124 | 0.0005583 | 1 | 1791 |
| 115.04 | 0.0006258 | 1 | 1598 |
| 113.06 | 0.0006947 | 2 | 2879 |
For each of the following questions, the Community Survey was grouped by respondents’ census tracts and an average score of each question was taken. All missing answers and “prefer not to answer” responses were removed from the data set prior to taking the average. 10 census tracts were then filtered out by lowest response, and plotted on the county map to indicate location and provide a visual cue to any geographic trends in the City.
Each question was asked according to a Likert scale, where 1 = “Extremely Dissatisfied” and 5 = “Extremely Satisfied”. The lower the response, the higher the tract was ranked in terms of risk.
To better understand what regions of the city are reporting lower-than-average levels of satisfaction with the City, we evaluated several questions from the Community Survey that recorded respondents’ sentiment about the city overall, as well as individual city services and resources. To begin, Question 1 was evaluated as a question to report overall quality of life in their neighborhood. For question 1, Census Tracts 19, 5 and 34 ranked lowest at scores of 2, 2.57, and 2.65, respectively.
Next, we look at Question 15 from the Community Survey, which asks respondents how well informed they feel with the City. From this analysis, we found that Census Tracts 117.04, 111, and 26 ranked lowest at mean scores of 2, 3, and 3.46, respectively. It is worth noting that these tracts are located outside of the downtown areas of the city, indicating that sentiments of involvement with the City decrease as you move away from the urban center.
Question 9C specifically asks respondents how easy it was to have their issues addressed with the city, which provides insight into the response time, level of completion, and quality of work being experienced by residents in the City.
Census Tracts 28, 24, and 1 had the three lowest responses at 2.5, 2.71, and 2.72, respectively.
Question 10_08 records responses about residents’ level of satisfaction with the enforcement of local ordinances and codes. Given that several of the most prevalent call topics are related to code enforcement topics, this indicates if there are areas of the city where these topics are more of a concern.
Census tracts 5, 9, and 21 reported the lowest levels of satisfaction with Code Enforcement, with average responses of 1.857, 1.875, and 1.888, respectively. Of note, Census Tract 21 has the highest number of calls to 311 per capita, indicating high usage of 311 services but relatively low levels of satisfaction with the services received.
Question 10_03 similarly asks about a service-specific task. Respondents are asked to rate their level of satisfaction with the quality of their trash collection service. From this question, we see that Tract 111 ranks lowest in this category, followed by 115.09. 34 also ranks lowly, along with a cluster of tracts near downtown (22, 21, 19, 20, and 6)- this could indicate that trash services experience more issues in rural areas as well as densely populated city centers, with better service being performed in areas with city infrastructure but a lwoer population density.
Question 20_3 asks respondents to provide their level of satisfaction with Trash Clean-Up enforcement. Given that Litter COde Violations are very large contributor to 311 calls in general, this question will provide insight as to how these issues are being handled & perceived by residents.
Census Tract 111 ranked lowest at an average score of 1, or as low as possible. Following 111, Census Tracts 34, 25, and 117.03 all report mean satisfactions of 2 or lower. Again, it appears that the west side of downtown and more rural, large tracts are reporting higher levels of dissatisfaction with trash clean-up.
Above, we evaluated the satisfaction across the City by highlighting census tracts ranking lowly across several questions in the community survey. Next, we seek to understand if there are specific census tracts that are repeatedly ranking in the bottom 5 census tracts to highlight regions that would benefit from investment across multiple services.
To perform this aggregation, we summarized ranked responses from Q1_01 (overall quality of life), Q10_03 (Satisfaction with trash collection), Q10_08 (Satisfaction with code enforcement), Q10_overall (Satisfaction with Overall maintenance of city), Q4 (Satisfaction with Availability of Information), Q9 (Satisfaction with Issues addressed by 311).
Next, each census tract was tallied to record the number of times that the tract was found in the bottom 5 tracts. The higher the tally, the more separate categories in which the tract underperformed. From this analysis, we found that Census Tract 111 is least satisfied across the selected fields; census tracts 21, 19, and 5 were also demonstrated to be at risk, forming a small cluster of interest just west of the downtown area.
This information above has also been provided in table format below.
Next, we aim to investigate 311 call prevalence across the city by first capturing the number of unique topics that are represented in the 311 call data. First, we evaluate the topic topics referenced across the city by tabulating the number of calls corresponding to organizational units within the city:
Next, we aimed to address the variable population across the City by separating call volume by census tract and calculating calls per capita by incorporating population data gathered in the 2020 Decennial Census. From this dataset, we are able to divide the number of calls by the population of each census tract to provide a per-capita level response for each call category.
Solid Waste calls comprise a large percentage of the total call topics placed with 311. This category includes topics such as requesting an extra trash pickup, reporting a missed trash pick-up, yard waste pick-up requests, and general concerns. We found that Census Tract 21 has the highest number of requests per capita at .38 requests per citizen. Census tracts 113.1, 117.04, 118.03, 118.04, and 119 did not report any solid waste reports to 311.
The number of calls placed to 311 by an individual regarding the same issue over time may indicate that services were not performed satisfactorily to the residents.
Here, we show the top 20 callers across the city in terms of numbers of calls placed to 311.
Next, we show the most frequent topics referenced by the repeat callers. Doing so required combining several similar call topic categories into one; for example, “Grass and Weed Neighborhood Services & Enforcement Violations” were combined with “Grass and Weed Code Violation” calls. Street light issues were the most frequent topic discussed by repeat callers, followed by Grass & Weed Code Violations and Vegetation Code Violations.
In order to understand if there are geographic trends surrounding repeat caller data, we plotted the individual request locations by user-provided latitiude and longtiude. Beginning with Street Light issues, we note that there are several clussters corresponding to roads in the area. Specifically, Lincolnway, W. Ewing Street, and Sample Street are among three of the most frequently referenced city streets.
We provide a closer view at the focus areas below:
A similar analysis was performed for grass and weed code violations. It was noticed that the 311 call topic changed from “Grass and Weed Code Violation” to “Grass and Weed Neighborhood Services” through the history of the 311 calls; both topics were merged for completeness.
First, we plotted each individual location by caller to understand the call locations and appreciate any clusters geographically.
Next, we aggregated the calls by Census Tract to evaluate call frequency as a trend across a larger area. From this analysis, we found that census tracts 4, 6, amd 20 were the tracts with the highest number of repeat callers; this region of the city just west of downtown had shown in prior analyses to be a hotspot for low community response scores, which makes the necessity for repeat calls to 311 for adequate service an interesting observation.
As the topic most frequently referenced in 311 calls overall, it was unsurprising to see a high number of repeat callers for this topic. We explored this data on the individual observation level by plotting occurrences across the City:
Next, each call was aggregated on the census tract level to track general geographic trends. Census Tract 31 ranked highest for number of litter-related repeat callers, followed by tracts 6 and 15, located adjacent to one another slighly northwest of downtown.
Next, we aimed to incorporate the Community Survey data into the Repeat Callers analysis to understand if tracts identified in the previous three sections are satisfied with the ease of contacting the city & having their addresses addressed. Below, we have included tracts 4,6,20,30,13,3.02,7,31,26,24,15,34,27,17,12, and 11- all having ranked the highest in repeat callers above- and calculated their mean response for question 9C of the community survey (“How Easy was it to Address Your Issue?), given that a respondent had answered”Yes” to having contacted 311 in the past year. The data showing distributions of the response to Q9B (“Have you contacted 311 in the past year?”) is indicated below geographically as well as by overall count. Compared to the city average response for Q9C of 3.54, 9 out of the 16 tracts of interest ranked lower than the city average.
Below displays the some of the demographic information provided by the American Community Survey by Census Tract. The toggle to the right filters the plots to examine demographic information by a particular Census Tract.
We identified the least satisfied census tracts by looking at the community survey responses and finding the tracts with the lowest average response to the question asking residents to rate the overall quality of life in their community. We excluded census tracts with fewer than five responses to the community survey and the results can be seen here:
| Census Tract | Average Quality of Life Rating |
|---|---|
| 5 | 2.571429 |
| 34 | 2.652174 |
| 23 | 2.727273 |
| 21 | 2.777778 |
| 20 | 2.875000 |
The demographics of these five least satisfied census tracts will be used for comparison to the overall demographics of South Bend to see if there are any glaring differences.
| Race | All of South Bend (%) | % in Most Dissatisfied Tracts | Difference |
|---|---|---|---|
| White | 58.1 | 34.6 | -23.5 |
| Black or African American | 25.2 | 43.9 | 18.7 |
| American Indian and Alaska Native | 0.5 | 1.2 | 0.7 |
| Asian | 2.0 | 0.1 | -1.9 |
| Native Hawaiian and Other Pacific Islander | 0.0 | 0.0 | 0.0 |
| Two or More Races | 8.6 | 10.6 | 2.0 |
| Other | 5.6 | 9.6 | 4.0 |
Looking at the table of race proportions in all of South Bend, compared to the least satisfied census tracts, there are some major differences. There is a large difference in the White population, with 23.5% less in the least satisfied census tracts. Conversely, there is a much higher proportion of Black or African American and Mixed Race residents.
| Ethnicity | All of South Bend (%) | % in Most Dissatisfied Tracts | Difference |
|---|---|---|---|
| Hispanic or Latino origin (of any race) | 5.8 | 24.9 | 19.1 |
Similar to the race demographics, when looking at the Hispanic/Latino population, we can see a much larger population concentrated in the least satisfied census tracts when compared to the entirety of South Bend. The difference is nearly 20%.
| Poverty Status (Relative to Poverty Line) | All of South Bend (%) | % in Most Dissatisfied Tracts | Difference |
|---|---|---|---|
| Below 100% | 21.4 | 32.2 | 10.8 |
| 100 to 149% | 11.8 | 22.9 | 11.1 |
| 150% or above | 66.8 | 44.8 | -22.0 |
Looking at the table on poverty status, we can see that the least satisfied census tracts have a much higher proportion of people below the poverty line, as well as a larger portion at 100 to 149% of the poverty line. Additionally, there is roughly 22% less residents that are 150% or more above the poverty line in the least satisfied census tracts compared to the whole of South Bend.
| Age | All of South Bend (%) | % in Most Dissatisfied Tracts | Difference |
|---|---|---|---|
| Under 5 years | 7.1 | 7.9 | 0.8 |
| 5 to 17 years | 20.0 | 25.2 | 5.2 |
| 18 to 24 years | 9.8 | 7.0 | -2.8 |
| 25 to 44 years | 27.8 | 25.6 | -2.2 |
| 45 to 54 years | 11.6 | 10.9 | -0.7 |
| 55 to 64 years | 10.4 | 12.1 | 1.7 |
| 65 to 74 years | 8.4 | 7.0 | -1.4 |
| 75 years and over | 5.6 | 4.4 | -1.2 |
Age is the first demographic where we do not see any significant differences in the proportions between the least satisfied and all of South Bend. This lack of notable differences could just be due to the granularity in the levels of age, and it is possible that if we made some larger age ranges with fewer groups then we would see larger gaps. The largest difference we can observe is in the 5 to 17 year old age group, with the least satisfied census tracts having about 5% more than the South Bend population.
There are still more demographics we could explore, but there were some clear insights from the few we checked out here. Further analysis may have additional findings to further support what we found here, but for the scope of our work we wanted to focus in on some of these key demographics of interest. While residents’ age did not seem to show any significant relationship with satisfaction, some of the other demographics showed significant differences in population proportions. To summarize, we found that the least satisfied census tracts had a much lower White population, with significantly higher Black or African American and Hispanic/Latino populations. There are far larger percentages of residents living in poverty, and far less residents living far above the poverty line in the least satisfied areas. From our analysis, it is impossible to determine causation and say what is truly causing the residents of these tracts to be dissatisfied, so we cannot attribute it to any one thing or any specific demographics. However, it is interesting to build more of a complete profile and see some of the commonalities between the census tracts that are unhappy with their quality of life. This can help the city of South Bend ideally better serve particular communities and understand the demographics and shared traits of the unhappy residents. There is much deeper analysis and insights that could be driven from this topic, but this is a good starting point to uncover the demographic disparities in satisfaction with the city.
In this section we will explore the intersection of 2022 South Bend 311 call volume by topic, resident satisfaction as expressed through the 2022 South Bend Community Survey, and resident desired city focuses and investment preferences also as expressed through the 2022 South Bend Community Survey.
In this section we chose to focus in on only information from 2022 across both the 311 calls data and the South Bend Community Survey as this was the most recent year of South Bend Community Survey survey data, and thus reflected the most recent resident satisfaction and investment preferences. We choose not to include past years’ South Bend Community Survey results as those results may have been representative of what residents wanted in the past and their inclusion in this section may not reflect efforts the city had already taken to address those previous resident concerns.
We will first explore the most frequent 2022 311 call topics by census tract in South Bend. This will provide insight into what residents are most frequently expressing concern about to the city through 311.
In the above plot we can see the most frequent 311 call topic for each census tract in 2022. Neighborhood service calls stand out as the 311 call topic that most frequently is the most called about in the majority of census tracts. We can also see that utilities calls seem to be a top concern for residents in census tracts further from downtown South Bend. We will next examine the resident’s expressed levels of satisfaction with major city services and see if there is a relation between the topics residents are calling into 311 about and their satisfaction with that service.
We will now explore the South Bend major city services that receive the lowest average resident satisfaction ratings expressed through the 2022 South Bend Community Survey by census tract. This will provide insight into what residents are least satisfied about and if it relates to the topics that residents are most frequently calling 311 about by census tract.
In the above plot we can see the lowest rated city service for each census tract in 2022, based on resident responses to question 10 of the 2022 South Bend Community Survey. We can see that city maintenance satisfaction is the lowest across the majority of census tracts in 2022. Local enforcement (otherwise termed neighborhood services) is represented as the lowest rated service in the second most census tracts.
There are also five census tracts which have a tie for lowest rated service and those can be found broken out below as their overlapping colors on the above chart do not accurately reflect their lowest rated service.
Census Tract: 35.00 Local Enforcement and City Maintenance
Census Tract: 111.00 Trash Service, Local Enforcement, and City Maintenance
Census Tract: 112.03 City Planning and City Maintenance
Census Tract: 113.09 Local Police, Local Enforcement, and Construction Permit
Census Tract: 117.04 Local Police, City Planning, Local Enforcement, Construction Permit, and City Maintenance
This analysis begins to show connections between frequent 311 call topics by census tract and city services that receive low resident satisfaction ratings, as we can see local enforcement as a frequently poorly rated service and a topic receiving frequent 311 calls. To further investigate this we will look into residents expressed focus / investment areas for the city.
We will now explore the top resident focus / investment preference by census tract expressed through the 2022 South Bend Community Survey. This will provide insight into what residents would like the city to devote focus to and if it relates to the topics that residents are most frequently calling 311 about and least satisfied about by census tract.
In the above plot we can see the most frequently expressed top city focus / investment area for each census tract in 2022, based on resident responses to question 11 of the 2022 South Bend Community Survey. We can see that city maintenance is a frequent priority across the majority of census tracts in 2022. Local police is represented as the second most frequent top resident focus.
Finally to put all of the analysis together we examined in a tabular format residents’ most frequent 311 call topic, lowest departmental satisfaction rating, and top three investment preferences by census tract.
| Cencus Tract | Most Frequent 311 Call Topic | Lowest Deprtmental Satisfaction Topic | MostCommon1stInvestment | MostCommon2ndInvestment | MostCommon3rdInvestment |
|---|---|---|---|---|---|
| 1.00 | UtilitiesCalls | CityMaintenenceSatisfaction | LocalPolice | CityMaintenence | ParksAndRec |
| 2.00 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | CityMaintenence | CityMaintenence | LocalEnforcement |
| 3.01 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | CityMaintenence | LocalPolice | NA |
| 3.02 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | LocalPolice | CityMaintenence | NA |
| 4.00 | NeighborhoodServicesCalls | LocalEnforcementSatisfaction | CityMaintenence | LocalEnforcement | CityMaintenence |
| 5.00 | NeighborhoodServicesCalls | LocalEnforcementSatisfaction | CityPlanning | CityMaintenence | CityMaintenence |
| 6.00 | NeighborhoodServicesCalls | LocalEnforcementSatisfaction | LocalPolice | LocalEnforcement | CityMaintenence |
| 7.00 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | LocalPolice | CityMaintenence | ParksAndRec |
| 8.00 | MayorOfficeCalls | ConstructionPermitSatisfaction | LocalPolice | CityPlanning | CityMaintenence |
| 9.00 | SolidWasteCalls | LocalEnforcementSatisfaction | CityMaintenence | CityPlanning | NA |
| 10.00 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | CityMaintenence | CityMaintenence | NA |
| 11.00 | NeighborhoodServicesCalls | CityMaintenenceSatisfaction | CityMaintenence | LocalEnforcement | NA |
| 12.00 | StreetsCalls | CityMaintenenceSatisfaction | LocalPolice | WaterService | CityMaintenence |
| 13.00 | StreetsCalls | CityMaintenenceSatisfaction | LocalPolice | CityMaintenence | NA |
| 14.00 | NeighborhoodServicesCalls | ConstructionPermitSatisfaction | CityMaintenence | CityMaintenence | NA |
When looking at the above sample of tabular analysis we see that residents have frequently rated their top call topic and least satisfied department in their top investment preferences. Thus, the city can utilize 311 call volume by census tract as a rough measure of the satisfaction of a particular census track with major city services.
Also we can see that across the community residents are calling in frequently for neighborhood services / local enforcement issues. Residents are also expressing the high dissatisfaction with neighborhood services via the 2022 South Bend Community Survey and are indicating it as a top priority for city focus and investment. To further investigate this and present specific actions the city can take in response to these concern we have identified top neighborhood services and city maintenance subtopics by tract using questions from the South Bend Community Survey that address more granular levels of the satisfaction with the services in question.
In the above plot we focus in on residents top priorities for Neighborhood Services and Enforcement. We can see that trash clean up enforcement is the top city focus / investment area for most census tracts in 2022. A second focus is business exterior maintenance which occurs frequently in census tracts closer to downtown.
In the above plot we focus in on residents top priorities for Traffic and Transportation. We can see that the condition of major city streets is the top city focus / investment area for most census tracts in 2022. There is not a stand out second focus, but adequacy of street lights and neighborhood snow removal both appear as top issues in at least four census tracts.
In the above plot we focus in on residents top priorities for Solid Waste. We can see that trash service charges are the top city focus / investment area for most census tracts in 2022. There is not a stand out second focus, but bulk item removal seems to be a priority in the north of the city while more central census tracts are more focused on improving the city’s efforts to keep them informed of disruptions to trash service.
In the above plot we focus in on residents top priorities for Public Safety. We can see that feeling safe in one’s neighborhood is the top city focus / investment area for most census tracts in 2022. The second focus would be enforcement of local traffic laws, while a third grouping is represented by tracts concerned about the relationship between police and the community. All three groupings are spread across the city with no apparent trend in location.
In the above plot we focus in on residents top priorities for Water. We can see that utility charges are the top city focus / investment area for most census tracts in 2022. The remaining topics are rather disjointed with no group making up a large majority, however, it is interesting to note that all tracts prioritizing the taste of their tap water as a focus for the city are located on the north west side of the city.
In the above plot we focus in on residents top priorities for Venues, Parks, and Arts. We can see that the maintenance of city parks is the top city focus / investment area for most census tracts in 2022. The second focus would be the condition of restrooms in public parks. It appears that restrooms are more generally cited as a top priority if the census track is closer to the downtown of the city.
Through the above analysis we have seen the relation between 311 call volume by topic, and resident satisfaction and investment preferences expressed through the South Bend Community Survey. It seems that residents frequently are dissatisfied with topics that they are frequently calling about and that residents frequently rate their highest call volume items as top areas for the city to improve, when viewed by census tract. Finally in an attempt to identify actionable insights in each census tract we examined residents top investment priorities within each major city service through the lens of the more granular survey questions within the Community Survey. In this examination we identified the top concerns that were echoed across census tracts and called out groupings of specific concerns that may be of interest to the city for further investigation.
Sentiment Analysis is a method used to computationally extract the underlying emotions or attitudes expressed in a piece of text. Sentiment analysis was performed on the CRM 311 in order to further analyze the South Bend residents’ satisfaction during their calls to the 311 call center. The analysis utilized two of the columns in the merged data from the Phone Call and Knowledge Article tables. The columns were the cosb_knowledgearticlename and the phone call description.
Analysis on the Knowledge Article name was performed first since the text has a shorter length and thus, requires less time and computational resources. Before getting into the sentiment piece, a quick exploratory analysis was performed were the frequency of each topic and character length were investigated. Below are the results.
| Knowledge Article Title | Frequency |
|---|---|
| Litter Code Violation | 4731 |
| Update Utility Account Information | 3831 |
| General Concern | 3618 |
| Street Light Issue | 3491 |
| Report Missed Trash Pick-Up | 2809 |
| Sewer Back Up | 2493 |
| Pothole in Street | 2329 |
| Grass and Weed Code Violation | 2288 |
| Dead Animal in the Street | 2255 |
| Customer Disputes | 2245 |
| Waterworks Service Request | 2023 |
| Alley Maintenance Request | 1990 |
| Parking Violations | 1956 |
| Abandoned Vehicles | 1697 |
| Trash Complaints | 1674 |
There are different lexicons that can be used to perform the analysts. A lexicon is a vocabulary of words that have been previously rated in terms of sentiment. The sentiment analysis package from the tidyverse package has 2 lexicons (bing and affin) that rate the words on a scale from positive to negative sentiment. Both were utilized on the Knowledge Article Title to asses which one was the most accurate at providing a sentiment.
The BING lexicon categorizes words in a binary fashion, either positive or negative.This lexicon was created by the Bing Liu and collaborators and was leveraged to derive the sentiment scores for the Knowledge Article title.
Look at the results above, we can see that there are an overwhelming number of negative words in the knowledge article titles.
The Knowledge Article Title was also run through the affin lexicon. The affin lexicon has a scale between -5 for negative word sentiment and +5 for positive word sentiment and was created by Finn Årup Nielsen between 2009 and 2011.
The overall sentiment using the affin lexicon was also negative.
The Affin Lexicon also attributed a negative sentiment to titles far more often than positive sentiments, however this lexicon fialed to assign a sentiment to a significant number of the words, making it not an ideal solution for this situation.
The BING lexicon performed better than the affin lexicon based on the amount of instances in which affin was not able to come up with a score and also common sense by looking over a few examples provided on the table below. A negative score indicates a negative sentiment, a 0 indicates neutral, and a positive value would indicate positive sentiment.
| Knowledge Article Title | Bing Sentiment Score | Affin Sentiment Score |
|---|---|---|
| Litter Code Violation | -1 | Unknown Sentiment |
| Update Utility Account Information | 0 | Unknown Sentiment |
| General Concern | -1 | Unknown Sentiment |
| Street Light Issue | -1 | Unknown Sentiment |
| Report Missed Trash Pick-Up | -2 | -2 |
| Sewer Back Up | 0 | Unknown Sentiment |
| Pothole in Street | 0 | Unknown Sentiment |
| Grass and Weed Code Violation | -2 | Unknown Sentiment |
| Dead Animal in the Street | -1 | -3 |
| Customer Disputes | 0 | -2 |
| Waterworks Service Request | 0 | Unknown Sentiment |
| Alley Maintenance Request | 0 | Unknown Sentiment |
| Parking Violations | 0 | Unknown Sentiment |
| Abandoned Vehicles | 0 | -2 |
| Trash Complaints | -2 | Unknown Sentiment |
To get s little bit more information to determine the sentiment of the call, sentiment analysis on the was also performed. Similar to the bing analysis, a first quick look at the column via exploratory analysis was done. Below is a sample of the Knowledge Article Titles with the most empty descriptions.
| Knowledge Article | Number of Articles | Empyty Descriptions | Percentage |
|---|---|---|---|
| Engineering General Concern | 386 | 148 | 38% |
| Create an Online Account for Utility Portal | 231 | 86 | 37% |
| New Trash and Yard Waste Service Information | 210 | 70 | 33% |
| Return Trip Customer Error | 246 | 79 | 32% |
| Request an Interior Code Inspection | 945 | 270 | 29% |
| Alley Maintenance Request | 1990 | 544 | 27% |
| No Article Found - Soilid Waste | 228 | 57 | 25% |
| Storm Damage Removal Policy | 278 | 69 | 25% |
| South Bend Home Repair | 765 | 182 | 24% |
| Abandoned Vehicles | 1697 | 386 | 23% |
Since there are some calls with no phone call description, the null values were filled in with the article topic whenever no description was available.
Due to computational constraints, the sentiment analysis on this column was only run using the BING lexicon. We chose the bing lexicon over the affin lexicon based on its past performance on the Knowledge Article Title analysis.
The output of the sentiment analysis from the phone call description was then joined to the census tract. The scores were then aggregated at a census tract level by averaging the net score of the result and counting the number of calls received by the 311 call centers based on different topics. An sample of the resulting data set can be observed below.
| Census Tract | Average Sentiment of Article Title | Average Sentiment of Article Description |
|---|---|---|
| 1 | -0.7372263 | -0.3211679 |
| 2 | -0.6324503 | -0.3509934 |
| 3.01 | -0.6943005 | -0.3108808 |
| 3.02 | -0.4415584 | -0.2987013 |
| 4 | -0.5707317 | -0.2536585 |
Based on this analysis, the census tracts with the top 10 high and low sentiment scores can be identified as shown by the tables below.
These 10 census tracts represent areas were the 311 call center and the city of South Bend have the largest area of opportunity to influence resident’s sentiment during a phone call and by consequence customer satisfaction with its services.
| Census Tract | Average Sentiment of Article Description | Total Calls |
|---|---|---|
| 27 | -0.7850467 | 321 |
| 20 | -0.7595819 | 287 |
| 5 | -0.7165992 | 741 |
| 19 | -0.6912929 | 379 |
| 34 | -0.6323529 | 884 |
| 31 | -0.5827620 | 1021 |
| 2 | -0.5672316 | 885 |
| 40 | -0.5297030 | 202 |
| 23 | -0.5166052 | 271 |
| 3 | -0.5119705 | 543 |
These 10 census tracts represent areas were the 311 call center and the city of South Bend have a smaller area of opportunity to influence resident’s sentiment during a phone call. These census are the ones with the most positive sentiment out of all the census tracts.
| Census Tract | Average Sentiment of Article Description | Total Calls |
|---|---|---|
| 17 | -0.1666667 | 366 |
| 11 | -0.1872000 | 625 |
| 58 | -0.2290503 | 179 |
| 54 | -0.2380952 | 63 |
| 1 | -0.2742475 | 299 |
| 59 | -0.2812500 | 96 |
| 41 | -0.2831858 | 226 |
| 10 | -0.2843137 | 306 |
| 16 | -0.2862069 | 580 |
| 14 | -0.2891986 | 287 |
Finally, the sentiment score was correlated with the number of calls from each census on different topics to see if there were any trends worth noticing.
The chart above shows there is a strong negative correlation between the number of calls regarding Neighborhood Service Call and average description sentiment. Meaning that the more calls a census tract get the lower the sentiment of the call tends to be. The opposite effect can be observed when it comes to calls regarding Facilities Ground Calls.
In conclusion, Sentiment analysis is a powerful technique that can serve as indicators of the emotions or moods expressed during the phone call that can give insight into areas of opportunities for improvement for the 311 Call Center and the City of South Bend as a whole. These analysis highlighted the areas of Neighborhood Services and Animal Resource Center Calls more specifically.
Our analysis focused on five key topics relevant to the CIty of South Bend: 311 usage and reporting behavior, demographic disparities within the city, resident investment preferences, trends over time in community survey results, and a sentiment analysis of 311 calls. Each of these topics was analyzed using a mixture of the city’s 311 calls data, the South Bend Community Survey data , and the American Community Survey (ACS) data. By looking at these separate datasets together we were able to identify trends and insights between each data source that may otherwise have gone unnoticed. Selected key findings from our analysis and actionable steps the city could take include:
There are limitations of our analysis that should be taken into consideration when analyzing our findings. When using 311 call data we made the assumption that the call location recorded for a 311 interaction is within the resident’s home census tract and that issues discussed on the call also pertained to their home census tract. While we believe this assumption is accurate for most cases, this may not always be the case as residents could call in from locations outside their census tract, may have outdated location information on record with 311, or may be calling from their census tract but calling about issues elsewhere in the city. We also faced practical limitations in the amount of data that we could work with. The South Bend Community Survey was used extensively in our work to identify resident satisfaction and investment preferences at a census tract level, however the survey has received only 600-700 responses when in the years it is administered. Based on this response rate and the number of census tracts examined across the city, it was not uncommon for our results at a census tract level to be based on an aggregation of 10 or less resident responses within a given census tract. This low level of responses available could mean that outlier opinions could skew the interpretation of resident opinions at a census tract level and should be considered when evaluating the results. Finally, when working with demographic data from the ACS we made the assumption that the demographics shown in the ACS data corresponded with the demographics of the residents providing South Bend Community Survey responses and 311 calls in a given census tract. We can not confirm that the demographics expressed in the ACS accurately reflect the makeup of the residents surveyed in the South Bend Community Survey for the census tract or the makeup of residents with the census tract making use of the 311 service.
Trevor Coville: trevorcoville@gmail.com
Cole Leppert: leppertcole@gmail.com
Helen Flynn: hmsflynn@gmail.com
Austin Kim: austinek94@gmail.com
Ana Dominguez: analauradmngz@gmail.com
Alexa Myers: amyers@resultant.com